{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e4f357d1-bbc9-4848-828c-00b36144ebdc",
   "metadata": {},
   "source": [
    "## Lesson 5: Text Generation with Vertex AI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a416b662-c645-4048-8557-fd0f66d9818c",
   "metadata": {},
   "source": [
    "#### Project environment setup\n",
    "\n",
    "- Load credentials and relevant Python Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9d7e7fc-0c16-41c5-ba3d-988fb011deb5",
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "from utils import authenticate\n",
    "credentials, PROJECT_ID = authenticate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff77bf9c-93e6-4281-9775-f7e87935b37a",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "REGION = 'us-central1'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02fb2abd-f5d3-4fa0-8c4c-8d516df20c7f",
   "metadata": {},
   "source": [
    "### Prompt the model\n",
    "- We'll import a language model that has been trained to handle a variety of natural language tasks, `text-bison@001`.\n",
    "- For multi-turn dialogue with a language model, you can use, `chat-bison@001`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4bc8793c-bae4-405e-9dcc-2d5657c7486b",
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "import vertexai\n",
    "vertexai.init(project=PROJECT_ID, \n",
    "              location=REGION, \n",
    "              credentials = credentials)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f253273a-a0bd-41c8-bf64-138666578504",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "from vertexai.language_models import TextGenerationModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da149416-7fea-4bb3-b872-f64200f9b19c",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "generation_model = TextGenerationModel.from_pretrained(\n",
    "    \"text-bison@001\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36ac4f25-7818-49fc-b9e9-097886da8e82",
   "metadata": {},
   "source": [
    "#### Question Answering\n",
    "- You can ask an open-ended question to the language model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3a1b5ab-1bb8-4882-93e8-cc9381b18aad",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "prompt = \"I'm a high school student. \\\n",
    "Recommend me a programming activity to improve my skills.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84aca913-e2c1-4ad5-b8ee-4945c9546102",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print(generation_model.predict(prompt=prompt).text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd540471-cda7-43b2-aee3-b84cb1234c1a",
   "metadata": {},
   "source": [
    "#### Classify and elaborate\n",
    "- For more predictability of the language model's response, you can also ask the language model to choose among a list of answers and then elaborate on its answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1491242-77fb-4d8e-8bda-04f4894fdd83",
   "metadata": {
    "height": 114
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\"I'm a high school student. \\\n",
    "Which of these activities do you suggest and why:\n",
    "a) learn Python\n",
    "b) learn Javascript\n",
    "c) learn Fortran\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28880988-e191-4991-baa8-fb7c89cc4874",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print(generation_model.predict(prompt=prompt).text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "913e102f-e4d7-4ac8-9b76-3a0c1b3790b9",
   "metadata": {},
   "source": [
    "#### Extract information and format it as a table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "efdfe0f7-678c-4ad6-bdcb-4cc299512570",
   "metadata": {
    "height": 352
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\" A bright and promising wildlife biologist \\\n",
    "named Jesse Plank (Amara Patel) is determined to make her \\\n",
    "mark on the world. \n",
    "Jesse moves to Texas for what she believes is her dream job, \n",
    "only to discover a dark secret that will make \\\n",
    "her question everything. \n",
    "In the new lab she quickly befriends the outgoing \\\n",
    "lab tech named Maya Jones (Chloe Nguyen), \n",
    "and the lab director Sam Porter (Fredrik Johansson). \n",
    "Together the trio work long hours on their research \\\n",
    "in a hope to change the world for good. \n",
    "Along the way they meet the comical \\\n",
    "Brenna Ode (Eleanor Garcia) who is a marketing lead \\\n",
    "at the research institute, \n",
    "and marine biologist Siri Teller (Freya Johansson).\n",
    "\n",
    "Extract the characters, their jobs \\\n",
    "and the actors who played them from the above message as a table\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92efacae-afe4-4cd2-9d6d-cb565d102ccc",
   "metadata": {
    "height": 63
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(prompt=prompt)\n",
    "\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d80be8ca-4732-4b39-a5e6-5db1d3b02218",
   "metadata": {},
   "source": [
    "- You can copy-paste the text into a markdown cell to see if it displays a table."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18816c8e-a473-4eb5-8983-7c5f80f8aa15",
   "metadata": {},
   "source": [
    "| Character | Job | Actor |\n",
    "|---|---|---|\n",
    "| Jesse Plank | Wildlife Biologist | Amara Patel |\n",
    "| Maya Jones | Lab Tech | Chloe Nguyen |\n",
    "| Sam Porter | Lab Director | Fredrik Johansson |\n",
    "| Brenna Ode | Marketing Lead | Eleanor Garcia |\n",
    "| Siri Teller | Marine Biologist | Freya Johansson |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87b9c406-2b3f-4fb0-a181-201091013864",
   "metadata": {},
   "source": [
    "### Adjusting Creativity/Randomness\n",
    "- You can control the behavior of the language model's decoding strategy by adjusting the temperature, top-k, and top-n parameters.\n",
    "- For tasks for which you want the model to consistently output the same result for the same input, (such as classification or information extraction), set temperature to zero.\n",
    "- For tasks where you desire more creativity, such as brainstorming, summarization, choose a higher temperature (up to 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f629e383-b4ad-4fa6-a949-435dd1f04da6",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "temperature = 0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84519df4-9b6f-471c-8183-27cc4a151d7d",
   "metadata": {
    "height": 63
   },
   "outputs": [],
   "source": [
    "prompt = \"Complete the sentence: \\\n",
    "As I prepared the picture frame, \\\n",
    "I reached into my toolkit to fetch my:\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a2fe53e-d885-411c-bb2e-c3940884d30f",
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(\n",
    "    prompt=prompt,\n",
    "    temperature=temperature,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2be86a8a-a58f-4b9e-bd66-9eced64e549c",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "print(f\"[temperature = {temperature}]\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac5bd8e5-8fdc-4d94-acf9-10b7d3b34ba8",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "temperature = 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0bf346d-2617-460e-bc47-f331b92ed053",
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(\n",
    "    prompt=prompt,\n",
    "    temperature=temperature,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67e25ae3-a1ae-4379-ad32-8d79e52b7590",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "print(f\"[temperature = {temperature}]\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fea806c4-f955-4434-89a5-57fd9083e9d3",
   "metadata": {},
   "source": [
    "#### Top P\n",
    "- Top p: sample the minimum set of tokens whose probabilities add up to probability `p` or greater.\n",
    "- The default value for `top_p` is `0.95`.\n",
    "- If you want to adjust `top_p` and `top_k` and see different results, remember to set `temperature` to be greater than zero, otherwise the model will always choose the token with the highest probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af98a491-6da8-4cca-bf16-85f9547e7fe5",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "top_p = 0.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e195211-60ea-4103-b083-19e78a26c920",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "prompt = \"Write an advertisement for jackets \\\n",
    "that involves blue elephants and avocados.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28093674-1a83-4abb-999a-030df75b2125",
   "metadata": {
    "height": 97
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(\n",
    "    prompt=prompt, \n",
    "    temperature=0.9, \n",
    "    top_p=top_p,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "30a4ce9d-7520-44bc-af24-dc1b6ad61b50",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "print(f\"[top_p = {top_p}]\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02817243-ef3f-440a-8846-33dc46963b41",
   "metadata": {},
   "source": [
    "#### Top k\n",
    "- The default value for `top_k` is `40`.\n",
    "- You can set `top_k` to values between `1` and `40`.\n",
    "- The decoding strategy applies `top_k`, then `top_p`, then `temperature` (in that order)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f6603255-1e19-47ce-8a77-634b33c1e99f",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "top_k = 20\n",
    "top_p = 0.7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef545757-b37c-4b39-b7e9-3ebc4ec1d910",
   "metadata": {
    "height": 114
   },
   "outputs": [],
   "source": [
    "response = generation_model.predict(\n",
    "    prompt=prompt, \n",
    "    temperature=0.9, \n",
    "    top_k=top_k,\n",
    "    top_p=top_p,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d09c4f8d-df9d-4d61-9ab6-19958f8b4129",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "print(f\"[top_p = {top_p}]\")\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "005c5ff2-03f0-4dcd-a808-c92c9c5bbb0a",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
